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Abstract 


This research aimed to evaluate students’ overall perceptions of Problem-based Learning use in a 
Computer Engineering program in Brazil. PBL is applied to nine interdisciplinary core courses of its 
curriculum. A 32-item questionnaire was administered to 115 students, with an average age of 20.9 (SD 
2.7) between late 2014 and the beginning of 2015. It includes items on PBL general aspects, improved 
Skills, PBL room and overall satisfaction. The results show that students’ degree of agreement with 
several items ranged from 4.1 to 5.9, in a seven-point scale. Some of these items revealed that 68.7% 
of the students got familiarized with PBL along the first year and there is a growing tendency among 
students to improve this familiarity with PBL over time. Moreover, the two items with lowest degree 
of disagreement were tutors’ feedback at the end of each tutorial session (35.7%) and the deadline to 
elaborate these solutions of problems (29.6%). Almost all students also agree with that PBL helps them 
to improve several skills, mainly problem-solving, self-directed learning and critical thinking. Repeating 
students in PBL courses scored higher than the non-repeating students. Many of them were statistically 
significant differences, inclusive for items on familiarization process. This can mean that their failures are 
not related to the familiarity with the method likely. Finally, most of the students are satisfied with PBL 
and like it, what indicates a successful use of it in the mentioned program. 
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Introduction 


Several authors have growly criticized the long-established teaching method in which 
one teacher speaks to many students. For Freire (1987) teachers “pour” knowledge in students 
as if they were empty containers, dividing classes between those who know and those who do 
not know, those who speak and those who listen, between those who prescribe and those who 
follow what was prescribed. Thus, it is necessary to create strategies for students to learn more 
actively. 

To address this problem, the Computer Engineering Undergraduate Program at the State 
University of Feira de Santana (UEFS) in Brazil, which has functioned since 2003, uses a 
hybrid model between long-established teaching method and Problem-based Learning (PBL). 
Although there are many ways of implementing PBL (Duch, Groh, & Allen, 2001), it has in 
common these characteristics: 1) a complex and realistic problem starts the learning process and 
motivates students in this; 2) collaboration among a students’ group — tutorial group — to discuss 
and solve the problem; 3) more time for self-directed learning of students to make choices about 
how and what they will learn and apply back to the problem; 4) teachers are facilitators of 
learning, promoters students’ active engagement with learning (Delisle, 1997; Savery, 2006). 
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In addition, PBL helps students to develop important and different skills: problem-solving, self- 
directed and lifelong learning, collaboration, effective communication, critical thinking, among 
others (Hmelo-Silver, 2004; Savery, 2006). 

Although Computer and Engineering courses are grounded in relevant theories, they 
have a very striking practical nature, so PBL suits very well to them and can make Computer 
Engineering more attractive for students. Hence, several institutions have implemented PBL in 
their Engineering and Computing programs and it has been widely accepted by teachers and 
students (AberSek, 2008; Mohd-Yusof, Hassan, Jamaludin, & Harun, 2011; Oliveira, Santos, & 
Garcia, 2013; Ribeiro & Mizukami, 2005). Moreover, PBL can help teachers to keep students’ 
attention without competing with technological distractions, one of the current challenges of 
education (Judd, 2014; Junco & Cotten, 2012; Sana, Weston, & Cepeda, 2013; Zhang, 2015). 
In an ethnographic study, the author observed PBL students spent little or no time using mobile 
devices during their meetings to discuss the problem while traditional students use them more 
often for non-academic purposes during their classes, especially to access Facebook (Santos, 
2012). In general, PBL students used mobile devices to take notes about their discussions — 
ideas, facts, issues and goals. 

In spite of the Computer Engineering program at UEFS has implemented PBL since 
it opened, in 2003, few evaluations of students’ perceptions of PBL were conducted and in 
addition they were informal, sporadic and very specific. Thus, the problem of this research 
is the lack of structured and overall evaluations of PBL in this program. Consequently, the 
research question of this work is: What is the PBL students’ overall impression of PBL method 
in their program? 

The aim of this research is to evaluate students’ general perception of PBL use in the 
Computer Engineering program at State University of Feira de Santana (UEFS) in Brazil. To 
this end, a questionnaire was administered to 115 students of this program. 


PBL at UEFS Computer Engineering 


UEFS Computer Engineering 1s a five-year program with 3955 curricular hours. Its 
curriculum contains nine courses based on PBL, which covers the core of Computer Engineering 
topics and totalizes 480 hours. Each PBL course has a conventional course associated with it 
at least. For instance, Programming PBL course has four co-requisites courses, 1.e. a student 
enrolled in Programming may also enroll in Algorithm and Programming II, Data Structure, 
Systems Design, Discrete Structure. These focus on theory while those focus on practice. This 
still means that PBL course integrates different disciplines, promoting an interdisciplinary 
approach, which can also be considered as a feature of PBL (Delisle, 1997). All PBL courses 
are listed in Table 1. 


Table 1. PBL courses in Computer Engineering program at UEFS. 


Semester (Term) PBL Courses Credit Hours Number of co-requisites courses 
a. Algorithms 30 1 
1 Programming 60 4 
2 Digital Circuits Design 30 1 
2° Software Engineering 60 3 
ya Concurrency and Connectivity 60 2 
a Digital Systems 60 1 
a Electronic Circuit Design 60 1 
4° Digital Signal Processing 60 2 
oO Compilers 60 2 
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In PBL courses, the students are gathered in small groups up to ten, maximum 
recommended value by literature, and each group is tutored by a teacher. To illustrate, in a 
freshman class, which has 40 students, they are divided into 4 groups containing 10 students 
at most and | tutor each one. During some tutorial sessions (classes), each group discusses the 
problem and proposes a solution, which in general is a program or a system design. The number 
of tutorial session varies according to the complexity of the problem. Among other tasks, tutors 
assess students’ performance during each tutorial session, their solutions and their technical 
reports on it. 

The remaining courses of this program are a priori based on traditional method in terms 
of having one teacher for one entire class. Why a priori? Because teachers can include active 
learning strategies for their own. There are numerous ways to do this. One simple example is 
to pause two or three times the class in order to students clarify their notes with partners. Thus, 
lecturers encourage students think about what they are learning. Straightforward strategies like 
this can obtain significant results (Ruhl, Hughes, & Schloss, 1987; Thaman, 2014). Inclusive, 
some teachers apply the own PBL in some of these courses sometimes. 


Methodology of Research 
General Background of Research 


This is a descriptive research in which a questionnaire was used as the main research 
instrument. The data were collected from a convenience sample of students enrolled in 
Computer Engineering program at UEFS between 2014 and 2015. 


Participants 


The convenience sample consisted of 115 students of almost 300 enrolled in Computer 
Engineering program at UEFS. One course by semester/term of the Computer Engineering 
program was visited in late 2014 to invite students to participate in this research, but many of 
these courses had already finished their activities or had few students. Due to this, some courses 
were revisited in the beginning of 2015. In addition, some students were invited in person out 
of classes. Even so, only approximately 40% of the students answered the questionnaire. They 
ranged from first-year to fifth-year students, with an average age of 20.9 (SD 2.7), 90.4% males 
and 9.6% females. While 75 of students never failed any course (65.2%), 20 repeated once 
(17.4%) and 20 repeated twice at least (17.4%). The Figure 1 shows the number of students 
had already completed different PBL courses. There is a high concentration of students up to 5 
PBL courses because many of them have dropped out of the program after this approximately. 


Frequency 
(number of 
students) 





Number of different PBL courses completed 


Figure 1: Number of students had already completed different PBL courses. 
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Instrument and Procedures 


This research used only one measure, Q-PBL, whose content validity and reliability 
were confirmed through a content expert committee, with overall Content Validity Index above 
.93, and Cronbach’s alpha above .77, respectively (Santos & Silva, 2015). It is a 32-1tem 
questionnaire to evaluate the general use of PBL by their students, according to these sections: 
PBL general aspects, improved skills, tutorial room and PBL overall satisfaction. Using a 
seven-point Likert scale, the participants’ agreements were measured through 15 statements 
related to PBL general aspects: interest in building own knowledge, problem (clarity and 
workload), tutor (feedback, evaluation and supervision), tutorial group (relationship among 
group members), tutorial session (utility) and familiarity with PBL. A ten-point Likert scale was 
used to measure the intensity with which participants realize some of their skills improved by 
PBL (verbal expression, writing, self-directed learning, problem-solving, planning, teamwork/ 
collaboration, critical thinking, reading comprehension and interpersonal relationships), the 
quality of elements of tutorial rooms (tables, chairs, whiteboards, air conditioning, size of 
the physical space and cleanliness) and the overall satisfaction with PBL use in their course. 
This research was approved by Institutional Review Board of our university. All participants 
assigned the consent form. 


Data Analysis 


The data analysis was carried out by means of the SPSS 20. The categories of the ordinal 
qualitative variables were associated with numbers in order to perform these calculations. 
Descriptive statistics was used to analyse means, standard deviations and percentages, while 
inferential statistics was conducted to compare different pair-groups: male and female, 
repeating students and non-repeating students, freshmen (n=37) and seniors (n=8). Freshmen 
were considered those, who completed one or two PBL courses only and seniors were those, 
who had more experience of the course and completed eight or nine different PBL courses. All 
pair-group scores were not normally distributed for one of their categories at least, as assessed 
by Shapiro-Wilk’s test (p < .05). Thus, Mann-Whitney U test was run to verify if there were 
statistically significant differences between the mentioned pair-groups. The null hypothesis is 
that the distribution of the analysed dependent variable is the same — or different for the 
alternative hypothesis — across each category of the analysed independent variable. This 
research only presents the results of the comparative analyses that were meaningful or showed 
statistically significant differences. 


Results of Research 


The Figure 2 shows the means of items of PBL general, which were all above 4.0. The 
three highest agreement means of the first section were “PBL method motivates me to search 
for my own knowledge”, “The problems are useful for my learning” and “Tutorial sessions help 
me to solve the problems”, respectively. The three lowest agreement means of PBL general 
aspect section were “Tutors give feedbacks on the tutorial group’s performance at the end of 
each tutorial session”, “The deadline to develop the solution is adequate” and “My tutorial 
groups generally have good and productive interpersonal relationships”, respectively. The 
percentages of disagreement with these three last 1tems in some degree were 35.7%, 29.6% and 
16.5%, respectively. 

An important issue is students’ familiarity with PBL method, which is addressed by the 
three last items in Figure 2. Most of the students agreed in some degree (68.7%) that they got 
familiarized with PBL along the first two terms (13.9% were indifferent and 17.4% disagreed 
in some degree). After these terms, this number increased from 68.7% to 81.2% of students 
(17.9% were indifferent and only 11.9% disagree in some degree). Moreover, 73.9% of the 
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students agreed in some degree that they like PBL (71% of first-year students and 75% of the 
remaining students). To analyse this issue deeper, students were grouped according to number 
of PBL courses completed. Those who completed one or two PBL course were grouped into 
first-year, those who completed three or four PBL courses were grouped into second-year, and 
so on. The groups approximately correspond to year of Computer Engineering course which 
students are enrolled. The result is shown in Figure 3. There are not any pattern in items 1.13 
and 1.15. The familiarization process and like or dislike PBL is a subjective matter. However, 
the item 1.14 in Figure 3 shows that there 1s a growing tendency among students to improve 
their familiarity with PBL over time. 

Figure 4 shows the results of skills (a) and tutorial room (b) sections, whose means were 
all above 7.1 and 6.9, respectively. Students scored higher problem-solving and self-directed 
learning and the remaining items differ little among them, varying approximately 0.6 at most. 
The lowest means were verbal expression, writing and collaboration (teamwork) skills. About 
tutorial rooms, where the tutorial sessions occur, the Figure 4(b) shows that their elements were 
well evaluated relatively, except chairs and whiteboard. 



























1.1. PBL method motivates me to go search for my own knowledge 


1.2. Tutorial sessions help me to solve the problems. 


1.3. My tutorial groups generally have good and productive interpersona 
relationships 


1.4. The number of students in each tutorial group is appropriate. 
1.5. The deadline to develop the solution is adequate 

1.6. The problems are clea 

1.7. The problems are useful for my learning 


1.8. Tutors contribute to progress tutorial sessions when necessar 


1.9. Tutors give feedbacks on the tutorial group’s performance at the end o 
each tutorial session 


1.10. Tutors make clear the criteria of solution evaluatio 


1.11. Solution evaluation performed by tutors is consistent with the defined 
criteria 

1.12. My performance evaluation performed by tutors is consistent with the 
defined criteria 

1.13. | got familiarized with PBL along the first two terms of my course 


1.14. Today | can say atl ee familiarized with PBL method [for those wha 
have already studied Computer Engineering for more than one year] 


1.15. | like PBL method 





Year 


Lyie 
2° 
[13° 
4° 
se 


Mean 





1.13. | got familiarized with PBL along 1.14. Today | can say that | get 1.15. | like PBL method 
the first two terms of my course — familiarized with PBL method [for those 
who have already studied Computer 
Engineering for more than one year] 


Figure 3: Students’ familiarity with PBL. 


Finally, the mean of PBL overall satisfaction, in the last close-ended item, was 7.3 (SD 
1.9). Only 26.1% students marked a value lower than 7. 
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The results show that there is a tendency for the men to score the items of PBL general 
aspects higher than the women (Figure 5) and to score the items of improved skills lower than 
the women (Figure 6). In Figure 5, the latter rated only the items on familiarity with PBL 
(1.13 and 1.14) higher than the former. Maybe they can feel more familiarized — or can be 
familiarized indeed — to PBL than the men. Nevertheless, only scores of items on tutors’ 
evaluation for males were statistically significant higher than for females: 1.10 (U=369.5, z=- 
1.967, p=0.49), 1.11 (U=369, z=-1.986, p=0.047) and 1.12 (U=361.5, z=-2.084, p=0.037). All 
statements of the items can be seen from Figures 2 and 4. 
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Figure 4: Means of items of skills and tutorial rooms sections. 


Mean 





11 42 413 14 (5 18 47% 48 419 140 4410 442 119 444 41.16 


Figure 5: Comparative means of items on PBL general aspects between male 
and female. 


Mean 





2.1 2.2 2.3 2.4 2.5 2.6 fof 2.6 Zi 


Figure 6: Comparative means of items on skills between male and female. 
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Seniors have a tendency to realize their skills improved more through PBL than 
freshmen (Figure 7). However, it was not found statistically significant differences between 
both categories. 

There is a curious tendency for the repeating students score the items of PBL general 
aspects higher than the non-repeating students, as can be seen from Figure 8. This tendency 
is stronger in the items on improved skills (Figure 9). These differences were statistically 
significant for the following items: 1.1 (U=993.0, z=-3.174, p=0.002), 1.3 (U=1,101.0, z=-2.398, 
p=0.17), 1.9 (U=1,169.5, z=-1.969, p=0.049), 1.13 (U=828.0, z=-4.05, p=0.0), 1.14 (U=523, 
Z=-3.431, p=0.001), 1.15 (U=1,107.5, z=-2.391, p=0.017), 2.1 (U=1,143.0, z=-2.132, p=0.033), 
2.3 (U=1,148.5, z=-2.114, p=0.035), 2.4 (U=977.5, z=3.151, p=0.002), 2.7 (U=1,145, z=-2.135, 
p=0.033). Among these differences, it 1s important to underline that repeating students felt 
much more familiarized to PBL than the others. In addition, a statistically significant difference 
(U=1020.5, z=-2.891, p=0.004) was also found between repeat (Mean 7.6, SD 1.8) and non- 
repeating students (Mean 6.7, SD 1.9) for the item on overall satisfaction. 


Mean 





2.1 2d Z.4 2.4 25 2.6 27 2.6 24 


Figure 7: Comparative means of items on skills between freshmen and seniors. 


Mean 





11642 13 74 18 16 47 48 419 1410 4.41% 442 1.13 114 1.15 


Figure 8: Comparative means of items on PBL general aspects between repeating 
and non-repeating students. 


— Repeating students 
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— —WNon-repeating students 


Mean 





2.1 2.2 | 2.4 2.5 2.6 ZF 2.6 28 


Figure 9: Comparative means of items on skills between repeating and non- 
repeating students. 
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Discussion 


The three items best evaluated in PBL general aspect section, in short, refer to self- 
directed learning and problem-solving skills, which were the two ones the students realized that 
they improved themselves by the method. They are also the two of main PBL skills stimulated 
by PBL according to several authors (Delisle, 1997; Duch et al., 2001; Savery, 2006). Thus, 
these results suggest that the main PBL goals have been reached by the program. 

Regarding to the three items with the lowest means in this section, the first sentence 
refers specifically to the feedbacks at the end of each tutorial session, but they do not need to 
occur every session. It is important that feedbacks on students’ progress were given frequently 
or periodically, in the last session of each problem, for instance (Boud & Feletti, 1998; Duch et 
al., 2001). The second item is a complex issue on deadlines. On the one hand, tutors elaborate 
a problem schedule based on their experience and its complexity. On the other hand, some 
students can have a pace of learning slower or difficulties to manage their time or tendencies to 
academic procrastinations (Kandemir, 2014). This issue needs to be verified closer. The third 
item evaluates two features at the same time, but the tutorial groups can have a good relationship 
and be productive and vice versa. As the item “Tutorial sessions help me to solve the problems” 
was better evaluated than it, maybe the relationships are being productive, but not so good. The 
collaboration skill was the third lowest mean of skills section. This might also be a reflection of 
the difficulty to deal with relationships. 

The other two items with the lowest means in the skills section were verbal expression 
and writing skills. However, their means were closer to other means, except solving-problem 
and self-directed learning skills which are the highest ones. Anyway, the most important fact 
is that students show improvements in all skills according to perceptions themselves. Some 
groups demonstrated a tendency to score higher than other groups, but this was more meaningful 
for repeating and non-repeating students, which had still statistically significant differences 
for four key skills in PBL — verbal expression, self-directed learning, problem-solving and 
critical thinking. A possible reason for this could be their effort to overcome failures or even to 
overcome themselves. Moreover, their failures in PBL courses are not related to the familiarity 
with the method likely. 

The two lowest means of tutorial room section are chairs and whiteboards. In fact, 
they need to improve their quality: the chairs are not so comfortable and some whiteboards 
are damaged. Whiteboards are widely used by students to note the main elements of their 
discussions, 1.e. ideas, facts, issues and goals. In spite of this seems a simple issue, this is often 
a complex and hard task for public university administration in Brazil for many reasons such as 
bureaucracy and limited funds. 

The results on students’ familiarity with PBL suggest that, in general, they got familiarized, 
even in their first-year. A reason that helps this is the PBL workshop which has been promoted 
since 2006 for freshmen when they enter the university, before the classes started. The aim 1s 
to prepare them to use this method, so the own PBL is used in order to students learn about 
PBL. In previous research, students had said that it indeed helps them (Moura & Santos, 2015; 
Soares, Luz, Santos, & Pinto, 2011). One can not necessarily say that repeating students failed 
in PBL courses due to no familiarization with it because they had degrees of agreement higher 
than non-repeating students for both first-year and following years. These repeating students’ 
degrees of agreement might be considered high because their means were greater than or 
equal to 6.0 in a seven-point scale. There is still a natural tendency to degree of agreement on 
familiarization increase over time, i.e. the more students use PBL, the more familiarized they 
got; even some of them do not like it for some reason. 

However, the results indicate that most of the students not only have got familiarized 
with PBL but also have liked it. Moreover, most of the students have also been satisfied with 
the benefits of PBL. This result is consistent with many authors. In short, in spite of contextual 
differences, students agree that PBL is better than traditional approach and is effective to 
improve skills like problem-solving (Ribeiro & Mizukami, 2005; Shamsan & Syed, 2009). 
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Conclusions 


This research filled a gap of lack of systematized evaluation of general perception of 
PBL use by students of a Computer Engineering program at UEFS. The results show that 
students have a good overall impression of PBL regarding to all evaluated items, which involve 
general features of PBL, perception on improved skill by PBL and infrastructure. About 
familiarization process, there is a growing tendency among students to perceive improvements 
in their familiarity with PBL over the time. Most of the students not only are satisfied with PBL 
but also like it. That is more frequent among repeating students proportionally. This research 
was very significant because it indicates a successful use of PBL in the Computer Engineering 
program at UEFS according to students’ perceptions. Finally, these results are of interest for 
all involved actors in the program like students, teachers and university administrators. They 
can contribute for both to reconfirm the way how PBL is implemented at UEFS and improve 
some aspects of this implementation. In this sense, the frequency of tutors’ feedback and the 
deadlines to deliver the solutions of problems need to be more investigated with the purpose 
of defining whether — or how — they can be improved. Thus, future works should focus on 
qualitative research to deepen these issues, if the interpersonal relationships are as good as they 
are productive and whether the identified difference between groups will corroborate or not 
these findings. Moreover, this kind of research allows to add the students’ visions the reasons of 
their own answers in Q-PBL. Finally, to carry out a longitudinal research can analyse changes 
of students’ perceptions over time. 
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